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Abstract 

Fifteen people recently graduated from a state-approved educational leadership program, 
K-12 School Administration specialization track, and sat for the Florida Educational 
Leadership Examination (FELE). Six of the fifteen graduates successfully completed a 
survey ranking courses in the program relative to their relevancy to the FELE. They also 
provided open ended responses, identifying most helpful and least helpful courses and 
information regarding same. Results yielded statistically significant differences across 
program courses relative to the FELE and qualitative information suggesting the need to 
continue to align program courses with the FELE, and the need for the State to test all 
leadership competency areas through its examination. 
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Study Purpose 

The study measured the perceived relevance and quality of graduate courses in an 
Educational Leadership program, as perceived by 2005 graduates. The purpose of the 
study was to identify courses that were most relevant and beneficial to program 
participants when taking the Florida Educational Leadership Examination (FELE) and to 
gather and analyze student opinions of particular program courses and their particular 
elements. 

Background and Rationale 

The FELE is an examination required by the Florida Department of Education 
(FLDOE) for licensure as a K-12 school administrator. Upon completion of relevant 
graduate course work, educators typically sit for the FELE as part of their requirements to 
enter school administration. Some students take the test while in program course work. 
FLDOE provides target outcomes for courses in State approved programs. Universities 
with leadership programs align courses in their programs with the provided standards. 

There is a range in how closely programs and courses align with prescribed 
outcomes across the eight domains: leadership, management, personnel, communications, 
curriculum, finance, law, and technology. Institutional assessment of such alignment 
often includes analysis of the percentage of passing grades in the courses, percentage of 
passing scores on the FELE by graduates, and feedback from school district leadership 
academy directors. In the spirit of providing an additional frame of measuring course 
relevance to the prescribed FELE outcomes, the Investigator surveyed recent graduates of 



one of these programs, to discover student perceptions of course relevance to the FELE. 
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Data Source and Study Delimitations 

Participants in the study were 2005 graduates of a state-approved Masters of 
Education in Educational Leadership program. The accessible population consisted of 15 
graduates from the School Administration specialization in the program who took the 
FELE. The setting for the program consisted of two campus locations and one in 
cyberspace. All participants were K-12 public school teachers aspiring to become school 
administrators. 

Method 

The study was one of mixed methodology measuring the elemental qualities and 
FELE relevance of courses in the program in quantitative and qualitative ways. Recent 
graduates received cover letters, informed consents, and surveys that were completed and 
returned anonymously in business reply envelopes. The survey consisted of a listing of 
the program courses whereby participants ranked and identified courses taken and 
responded to two open ended questions: What elements of your top and bottom ranked 
courses helped you most and least with the FELE. 

The Investigator posed the following null hypothesis: 

There will be no significant difference across the means of course relevancy rankings, 
when it comes to the perception of them by program graduates. 

Survey Response 

Ten of fifteen surveys were returned. Nine surveys had been completed and one 
survey was returned due to the study participant moving and possessing an expired 
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forwarding address. Three of the nine completed surveys were disqualified in the 
quantitative ranking portion due to participants providing multiple rankings of l’s and 
12’s, rather than using all numbers from 1 through 12 as instructed. Six surveys were 
correctly completed and used in the study (see Appendix A for survey example). 

Analysis of Quantitative Data 

The Investigator conducted an Analysis of Variance (ANOVA) of the participant- 
ranked courses. The F ratio (F=(found variation of the group averages)/(expected 
variation of the group averages) revealed standard deviations of the course mean scores 
greater than the proposition of the null hypothesis where no such significance exists (F 
typically = 1.0 for the null). The F Ratio was 3.846. Therefore, the null hypothesis was 
rejected and it was concluded that there were statistically significant differences across 
courses and their relevance to the FEFE. 

A t-test had been considered for analysis. However the sheer number of groups 
(courses) nullified the value of a t-test because of the number of possible pairings of 
courses, and the less than desired value of one pair among many being significantly 
different (P=.05 for one pair of courses). The ANOVA aggregated the data into one 
number (F) and provided one P for the null hypothesis which led to its rejection. 

ANOVA Results 



Source of 


Sum of 


d. f . 


Mean 


F 


Variation 


Squares 




Squares 




between 


335.7 


11 


30 . 52 


3 


error 


476.2 


60 


7 . 936 




total 


811 . 9 


71 







The probability of this result, assuming the null hypothesis, is 0.000 
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EDU 605 Intro to Educational Administration: 

Number of items= 6 

6.00 9.00 11.0 11.0 11.0 11.0 

Mean = 9.83 

95% confidence interval for Mean: 7.533 thru 12.13 
Standard Deviation = 2.04 
Hi = 11.0 Low = 6.00 
Median = 1 1 .0 

Average Absolute Deviation from Median =1.17 



EDU 621 Human Resources: 

Number of items= 6 

5.00 7.00 8.00 10.0 10.0 10.0 

Mean = 8.33 

95% confidence interval for Mean: 6.033 thru 10.63 
Standard Deviation = 2.07 
Hi = 10.0 Low = 5.00 
Median = 9.00 

Average Absolute Deviation from Median = 1.67 



EDU 622 Cultural Diversity: 

Number of items= 6 

5.00 6.00 6.00 8.00 11.0 12.0 

Mean = 8.00 

95% confidence interval for Mean: 5.699 thru 10.30 
Standard Deviation = 2.90 
Hi = 12.0 Low = 5.00 
Median = 7.00 

Average Absolute Deviation from Median = 2.33 



EDU 630 Transformational Leadership and Innovation: 

Number of items= 6 

3.00 3.00 5.00 8.00 9.00 10.0 

Mean = 6.33 

95% confidence interval for Mean: 4.033 thru 8.634 
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Standard Deviation = 3.08 
Hi = 10.0 Low = 3.00 



Median = 6.50 

Average Absolute Deviation from Median = 2.67 



EDU 635 Technology and Administration: 

Number of items= 6 

1.00 4.00 5.00 7.00 8.00 9.00 

Mean = 5.67 

95% confidence interval for Mean: 3.366 thru 7.967 
Standard Deviation = 2.94 
Hi = 9.00 Low = 1.00 
Median = 6.00 

Average Absolute Deviation from Median = 2.33 



EDU 699 Internship: 

Number of items= 6 

4.00 7.00 8.00 10.0 12.0 12.0 

Mean = 8.83 

95% confidence interval for Mean: 6.533 thru 11.13 
Standard Deviation = 3.13 
Hi = 12.0 Low = 4.00 
Median = 9.00 

Average Absolute Deviation from Median = 2.50 



EDU 624 Communication S ki lls: 

Number of items= 6 

2.00 4.00 4.00 4.00 6.00 9.00 

Mean = 4.83 

95% confidence interval for Mean: 2.533 thru 7.134 
Standard Deviation = 2.40 
Hi = 9.00 Low = 2.00 
Median = 4.00 

Average Absolute Deviation from Median = 1.50 



EDU 625 Educational Finance: 
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Number of items= 6 
E00 E00 E00 2.00 2.00 12.0 

Mean = 3.17 

95% confidence interval for Mean: 0.8661 thru 5.467 
Standard Deviation = 4.36 
Hi = 12.0 Low = 1.00 
Median =1.50 

Average Absolute Deviation from Median = 2.17 



EDU 623 Educational Law: 

Number of items= 6 

1.00 1.00 2.00 2.00 2.00 6.00 

Mean = 2.33 

95% confidence interval for Mean: 3.2812E-02 thru 4.634 
Standard Deviation =1.86 
Hi = 6.00 Low = 1.00 
Median = 2.00 

Average Absolute Deviation from Median = 1.00 



EDU 670 Special Education for Administrators: 

Number of items= 6 

4.00 6.00 7.00 7.00 8.00 8.00 

Mean = 6.67 

95% confidence interval for Mean: 4.366 thru 8.967 
Standard Deviation =1.51 
Hi = 8.00 Low = 4.00 
Median = 7.00 

Average Absolute Deviation from Median = 1.00 



EDU 698 Practical Principal: 

Number of items= 6 
3.00 5.00 8.00 9.00 10.0 11.0 

Mean = 7.67 

95% confidence interval for Mean: 5.366 thru 9.967 
Standard Deviation = 3.08 
Hi = 11.0 Low = 3.00 
Median = 8.50 

Average Absolute Deviation from Median = 2.33 
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EDU 626 Curriculum and Instruction Leadership: 

Number of items= 6 

3.00 3.00 3.00 7.00 9.00 10.0 

Mean = 5.83 

95% confidence interval for Mean: 3.533 thru 8.134 
Standard Deviation = 3.25 
Hi = 10.0 Low = 3.00 
Median = 5.00 

Average Absolute Deviation from Median = 2.83 



Course Relevancy to FELE (from most relevant to least relevant): 



Course 


Mean 


SD 


EDU 623 School Law 


2.33 


1.86 


EDU 625 Educational Finance 


3.17 


4.36 


EDU 624 Communication Skills 


4.83 


2.40 


EDU 635 Technology and Admin 


5.67 


2.94 


EDU 626 Curriculum & Instruction 


5.83 


3.25 


EDU 630 Transformational Leadership 


6.33 


3.08 


EDU 670 Special Ed for Administrators 


6.67 


1.51 


EDU 698 Practical Principal 


7.67 


3.08 


EDU 622 Cultural Diversity 


8.00 


2.90 


EDU 621 Human Resources 


8.33 


2.07 


EDU 699 Internship 


8.83 


3.13 


EDU 605 Intro to Ed Administration 


9.83 


2.04 



Qualitative Component 



Beyond the quantitative portion of the study, participants provided answers to two 



open- ended questions and identified the three courses that were most and least relevant 



to their FELE experience: 
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(n of 6) 
Course 
EDU 623 
EDU 625 
EDU 624 
EDU 635 
EDU 626 
EDU 630 
EDU 670 
EDU 698 
EDU 622 
EDU 621 
EDU 699 
EDU 605 



# of markings for most relevant 

3 

4 
1 
0 
2 
2 
1 
1 
0 
1 
0 
0 



# of markings for least relevant 
0 
1 
0 
1 
2 
1 
0 
1 
2 
1 
4 
3 



Standard deviations correlated with courses identified in open responses as both 
most and least relevant to the FELE. In several cases courses were identified as most and 
least relevant to the FELE by the study participants suggesting a substantial range of 
perceptions of program courses and their relevancy to the FELE across study participants. 

A sizable standard deviation was noticed in EDU 625 Educational Finance as it 
was identified as very relevant by five participants and least relevant by one participant, 



with supporting narrative for same. 




FELE 11 



Other considerable standard deviations existed in courses that were taught by 
more than one instructor suggesting dissonance across sections of courses. More research 
is needed to consider this hypothesis. 

The Investigator had knowledge of the degree of alignment of each course in the 
program to the FELE anticipated outcomes, during the study, and saw a correlation 
between those courses aligned to the FELE and their participant-perceived relevance to 
the FELE. More research is needed to consider this conjectural conclusion. 

Open-Ended Questions 

Participants provided answers to two open-ended questions: 

1. What elements of top ranked courses helped you most with the FELE? 

2. What elements of bottom ranked courses helped you least with the FELE? 

Regarding question one, participants shared comments regarding top-ranked 

courses, 

instructors, and course delivery: 

Participant 1: 

“The instructor for EDU 625 was one of the most helpful. He would key us into 
possible FELE items during lectures and tell us what needed to be studied to guarantee 
success. On the FELE EDU 625 was the most helpful! The instructor did a great job 
highlighting items we would need. Another instructor highlighted particular information 
we needed on the FELE as well. It was a very informative class also.” 
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Participant 2: 

“Most of the FELE questions reference Law, Finance, and the Human Resource 
aspects of being an administrator. I assume these are among the most important skills 
needed to maintain a consistent leadership style. It helped on the test but in my opinion I 
don’t believe the information had much relevance to the everyday job.” 

Participant 3: 

“The attributes of those courses that caused them to be the most helpful are: 
Quality instruction, material content, and intelligent interactions with peers (referring to 
EDU 624, EDU 626, and EDU 625).” 

Participant 4: 

“I do feel that these classes were well organized (EDU 623, EDU 670, and EDU 
626) and gave real insight upon encouraging true leadership skills of a school based 
Principal. I particularly enjoyed a seminal text in the curriculum course. The class was a 
special group in which one day we will all encourage these life and educational lessons 
brought forth by this experience at the university.” 

Participant 5: 

“Courses (EDU 625, EDU 623, EDU 630) correlated with the FELE components. 
Relevant information. Direct-live-consistent instruction/feedback.” 

Participant 6: 

“Information by instructors clearly communicated (EDU 625, EDU 623, EDU 
630). Objectives coincided with textbook and items relevant to FELE were emphasized.” 
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Regarding question two, participants shared comments regarding lowest-ranked 

courses, instructors, and course delivery 
Participant 1: 

“I don’t recall information being on FELE from the above classes (EDU 699). I 
am sure it was but considering my situation-my dad’s funeral 3 days prior to test. I don’t 
recall many specifics of test. I don’t feel I can pick a total of three least helpful (courses). 
I took and passed FELE early on in my program. During remaining courses I noticed 
Profs making FELE a priority.” 

Participant 2: 

‘‘These seemed l ik e fluff courses (EDU 622, EDU 630, EDU 605) to me. 

We leam a lot of theory that doesn’t really help with daily procedures or tasks. I don’t 
feel any wiser on the subject of leadership after completing these courses.” 

Participant 3: 

“The attributes of these courses (EDU 698, EDU 621, and EDU 635) that caused 
them to be least helpful were none.” 

Participant 4: 

“Firstly, I felt that EDU 625 did not provide enough information about finance. 
The rest (EDU 622, EDU 699) were not included in the test; however there were great 
classes and I learn(ed) a bulk of new information.” 

Participant 5: 

“There was no clear correlation between course objectives (EDU 699, EDU 605, 
and EDU 626) and components of the FELE.” 
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Participant 6: 

“Information did not align with components covered on FELE (EDU 699, EDU 
605, and EDU 626). No real reference to how course work would help prepare students 
for FELE.” 

Qualitative Data Analysis 

Through the quantitative portion of the study it was discovered that there are 
differences across program courses in relation to their quality and relevance to the FELE, 
as perceived by recent graduates of the program. 

Participant qualitative data suggested the FELE examination is school law and 
school finance heavy. More than one respondent reported that not all of the eight areas of 
prescribed FELE outcomes were evident on the examination. Half of the respondents 
said that several courses in the program did not seem to be aligned with the examination. 
One participant complained about theory in course work and felt that only practical 
information should be provided. Another participant shared concern about the emphasis 
on only a few topics (law, finance, and human resources) on the examination and 
questioned the validity of these elements being essential to day to day school leadership 
duties and responsibilities. 

Although there were eight core areas prescribed by FELE objectives the data 
suggested that not all areas were evident on the examination. More research is needed to 



consider this conclusion. 
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Summary 

Study results will be used to enhance courses. Misaligned courses, and their 
elements, identified by respondents as being less than helpful will be diminished in the 
program to better prepare program graduates for the FELE. Results of, and for, the high 
ranking courses will be sustained and strategies to synthesize positive attributes of 
courses to courses currently identified as less helpful will be implemented. 

The qualitative portion of the study provided thoughtful descriptions of course 
elements that illuminated contextual and content attributes of courses in the program. 
The shared elements that are content based, and desired, will guide the program in further 
connections between the courses and the FELE. More research is needed to particularly 
identify courses elements to discover how they particularly aligned with the prescribed 
FELE outcomes. 

Study Limitations 

The study sample is one of convenience, in that study participants are those who 
were recent graduates of the program and the results are applicable only to the particular 
program. Still, the results can be contemplated by all state-approved Educational 
Leadership programs and others interested in program evaluation. Adding the student 
perception component of evaluation to existing program evaluation techniques has the 
potential to enhance any program. 

Implications 

Universities with state-approved Educational Leadership programs have invested 
significant time in aligning program courses with FELE proficiencies. Typically program 




FELE 16 



evaluation involves analyses of passing grades, performance on the FELE, and external 
feedback from school district leadership academy directors. The added evaluation piece 
of analyzing substantive feedback from program graduates provides an additional way of 
seeing the intended connections between program courses and the high-stakes 
examination. 

New beginning Principal proficiencies were proposed for Florida Educational 
Leadership programs in 2004. The new proficiencies were officially adopted in April of 
2005. State-approved Educational Leadership programs will now redesign courses in 
their programs to align with the new FELE proficiencies. 

Various layers of people involved in educational leadership statewide are now 
proposing components of the new FELE examination. Adding the participant-perceived 
piece to educational leadership program evaluation provides data for revising program 
courses. Data also aids in aligning courses to the new FELE proficiencies: Feedback from 
participants regarding the elements of the new test will be helpful in revising program 
courses so they are better aligned with the new FELE outcomes. 

The results of this study suggest that more needs to be done to align program 
courses with prescribed FELE outcomes. Results also suggest that the FLDOE needs to 
provide a more comprehensive examination covering all prescribed FELE proficiencies. 
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Appendix A 



Rank the following courses (1 being most helpful, 12 being least helpful) in regard to 
how much the course prepared you for the FELE examination: 



My campus location: Boca Raton 



Royal Palm Beach 



Course Ranking 

EDU 605 Introduction to Educational Administration 

EDU 621 Human Resource Management & Development 

EDU 622 Cultural Diversity in Educational Systems 

EDU 630 Transformational Leadership Skills 

EDU 635 Technology and Administration 

EDU 699 Internship 

EDU 624 Communication Skills 

EDU 625 Educational Finance 

EDU 623 Educational Law for Sch Admin 

EDU 670 Special Education for the Admin 

EDU 698 The Practical Principal 

EDU 626 Public School Curriculum & Instruction 



(Ranking= 1 being most helpful, 2, 3, etc, 12 being least helpful) 



Identify the three least helpful courses in your preparation for the FELE exam: 



? ? • 

What attributes of these courses caused them to be least helpful in your preparation for 



the FELE? 



(attach an additional sheet of paper for comments if you need to) 



Identify the three most helpful courses in your preparation for the FELE exam: 



? ? • 

What attributes of these courses caused them to be most helpful in your preparation for 



the FELE? 



(attach an additional sheet of paper for comments if you need to) 





Appendix B 
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Informed Consent 

A Study of Participant-Perceived Quality of Courses and FELE Relevance in a Florida 
Graduate Educational Leadership Program 

Please read this consent document carefully before you decide to participate in this 

study. 

Dear Lynn University M.Ed. Educational Leadership Program Graduate, 

As a recent graduate of Lynn University, you are invited to participate in “A 
Study of Participant-Perceived Quality of Courses and FELE Relevance in a Florida 
Graduate Educational Leadership Program.” 

The study investigator is Dr. David Moffett, Associate Professor & Coordinator 
of Med/PhD in Educational Leadership, Ross College of Education, Lynn University, 
Boca Raton, Florida 33431. 

The study is one of mixed methodology whereby the investigator shall collect, 
analyze, and report upon quantitative and qualitative data. The research shall be 
conducted from August 25th through October 5th, 2005. 

Purpose of the research study: 

The purpose of the research is to collect and report upon participants’ perceptions 
of quality of courses and their relevance to the Florida Educational Leadership 
Examination (FELE) in the M.Ed. Educational Leadership program. 

What you will be asked to do in the study: 

Participants are asked to complete and return a survey containing two likert scales 
related to the particular courses they experienced in their program (12 in all) and provide 
four short answers to prompts asking them to identify what elements of courses listed 
caused them to be most and least effective, in anonymous form. Participants shall retain 
the received copy of the consent form for their personal records and return the 
anonymous survey at their earliest convenience in a provided business reply envelope. 

Time required: 

The amount of time a participant can plan on investing in the completion of the 
survey is 15 to 30 minutes. Participants shall determine the most practical location for 
completion of the survey. 

There are no alternative procedures or treatment that may be advantageous to 
participants in this study. 

The survey procedure shall be self-administered by study participants and then 
sent to the study investigator in the provided envelope. 
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Risks and Benefits: 

There are no anticipated risks or discomforts resulting from participation in the 
study. While there are no potential benefits to study participants, there are potential 
benefits to current and future students in the program. Data provided by study 
participants shall be analyzed and study results will inform program course quality and 
FELE relevance. 



Anonymity: 

Completed surveys shall only be reviewed by the study investigator. 

Surveys shall be kept in a locked file cabinet for up to five years and then they will be 
destroyed. Your identity will be kept anonymous to the extent provided by law. 

Compensation: 

There will be no compensation awarded for participation in the study. 

Study participants have the right to withdraw consent for participation in the study at any 
time without consequence. 

Voluntary participation: 

Study participants do not have to answer any question that s/he does not wish to 
answer in the provided survey. There will be no tape recordings or other electronic means 
of gathering data. Your participation in this study is completely voluntary. There is no 
penalty for not participating. 

Study results will be shared via a planned presentation at the Florida Educational 
Research Association’s Annual Meeting in 2005, and subsequent publications by the 
investigator. Results will also be shared with Lynn University’s Vice President of 
Academic Affairs, Academic Dean and Dean of the Ross College of Education. Study 
participants may request an electronic copy of the presentation paper via an electronic 
mail request to the investigator at dmoffett@lynn.edu after November 30, 2005. 

You may contact the study investigator, by phone or by e-mail, at any time with 
any questions. The investigator is Dr. David Moffett and his phone number is 
561.237.7840. His e-mail address is dmoffett@lynn.edu . 

You may contact an IRB Representative with any questions you may have at any 
time. The Chair of the IRB is Dr. Farideh Farzamand and her phone number is 
561.237.7847. Her e-mail is FFarazmand@Lynn.edu . 

On behalf of the students and faculty in the M.Ed. Educational Leadership 
program at Lynn University, I want to thank you for your consideration to participate in 
this study. 



Consent 
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By completing the enclosed survey and returning it in the provided business reply 
envelope you imply consent for participating in the study. Please retain this consent form 
for your records. 




